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The  Line  Operations  Safety  Audit  Program:  Transitioning  From  Flight 
Operations  to  Maintenance  and  Ramp  Operations 


INTRODUCTION 

Managing  risks  has  become  increasingly  important 
in  modern  organizations,  including  medicine,  aviation, 
and  finance.  The  initial  identification  and  interpretation 
of  hazards  are  some  of  the  most  challenging  aspects  of 
risk  management,  since  many  hazards  remain  hidden, 
unnoticed,  or  misunderstood  for  long  periods  of  time 
before  an  accident  (Macrae,  2009;  Turner,  1994).  The 
risks  associated  with  these  hazards  seem  obvious  after  an 
accident;  however,  the  early  signs  pointing  to  an  emerg¬ 
ing  hazard  and  its  consequent  risk  are  often  extremely 
weak  and  ambiguous  (Reason,  1997;  Vaughan,  1996). 

Systems  such  as  the  National  Aeronautics  and  Space 
Administration  (NASA)’s  Aviation  Safety  Reporting 
System  (ASRS)  and  the  Maintenance  Aviation  Safety 
Action  Program  (M-ASAP)  encourage  air  carrier  and 
repair  station  employees  to  voluntarily  report  certain 
safety  information.  These  programs  provide  an  important, 
previously  unavailable,  source  of  data  that  is  captured 
rapidly  and  directly  from  those  responsible  for  the  day- 
to-day  safe  operation  of  the  aviation  system.  However, 
systems  like  these  are  used  proactively  and  are  based  on 
previous  adverse  events. 

A  Line  Operations  Safety  Audit  (LOSA)  is  a  tool  for 
collecting  safety  data  during  normal  airline  operations. 
As  a  voluntary  safety  program,  a  LOSA  does  not  require 
Federal  Aviation  Administration  (FAA)  approval,  ac¬ 
ceptance,  or  monitoring,  as  stated  in  the  FAA  LOSA 
Advisory  Circular  120-90  (2006).  The  agreement  allows 
the  air  carrier  or  air  agency  to  maintain  control  of  the 
audit  results.1 

Monitoring  routine  operations,  the  cornerstone  of  the 
LOSA  process,  addresses  an  important  aspect  of  safety 
auditing,  namely,  that  risks  and  human  error  can  never 
be  completely  eliminated.  Recognizing  correct  and  incor¬ 
rect  actions  to  manage  these  risks  and  errors  before  they 
manifest  into  larger  incidents/accidents  makes  LOSA 
a  truly  proactive,  rather  than  a  reactive  strategy,  as  well 
as  a  workable  predictive  way  of  risk  mitigation  (ICAO, 
2009;  Maurino,  2001).  Several  companies  have  instituted 
LOSA  programs  and  have  garnered  many  valuable  les¬ 
sons,  safety  improvements,  and  significant  returns  on 
their  investment. 


The  FAA  ATA  Human  Factors  Taskforce  has  discussed  referring  to 
LOSA  within  maintenance  and  ramp  operations  as  an  assessment 
rather  than  audit  to  reflect  the  nonpunitive  intent  of  the  program. 


LOSA  has  evolved  into  a  strategy  comprised  of  sys¬ 
tematic  line  observations  of  routine  operations  to  provide 
safety  data,  both  in  the  technical  and  human  performance 
areas.  During  a  LOSA  observation,  observers  record  and 
code  potential  threats  to  safety,  how  the  threats  were 
addressed,  the  errors  generated,  how  the  errors  were 
managed,  and  how  the  observed  behaviors  could  be 
associated  with  incidents  and  accidents.  The  data  from 
LOSA  observations  provide  indicators  of  organizational 
strengths  and  weaknesses,  which  facilitate  the  develop¬ 
ment  of  countermeasures  to  operational  threats  and  errors 
(ICAO,  2002).  Prior  to  the  implementation  of  LOSA, 
safety  analysis  of  the  effect  of  human  performance  in  avia¬ 
tion  had  been  retrospective,  thus  overlooking  processes 
immediately  preceding  the  human  error  that  resulted 
in  incidents/accidents  (Maurino,  2001).  Investigators 
targeted  those  actions  and  inactions  that  did  not  pro¬ 
duce  desired  outcomes,  often  without  fully  considering 
the  mismanagement  of  processes  leading  to  these  safety 
breakdowns. 

Accident  investigation  concentrates  on  failures,  which 
are  important  for  discovering  major  breakdowns  in  the 
system,  but  failures  are  rare  events.  Self-reporting  of  in¬ 
cidents  and  potential  hazards  preceding  major  accidents 
can  be  limited  because  personal  biases  about  behavioral 
norms  may  result  in  overlooking  significant  actions,  and 
there  are  always  concerns  about  professional  consequences. 

In  addition  to  flight  deck  operations,  there  is  a  need 
to  study  aviation  maintenance  and  ramp  operations  from 
a  neutral  perspective  during  normal  operations.  Mainte¬ 
nance  organizations  and  ground  operators  have  the  op¬ 
portunity  to  benefit  from  the  10-year  success  of  normal 
operations  audits  on  the  flight  deck.  LOSA  provides  a 
minimally  invasive  safety  audit  of  maintenance  and  ramp 
operations  to  evaluate  an  organization  (including  its  sys¬ 
tems,  processes,  and  personnel),  ascertain  the  validity  and 
reliability  of  its  information,  and  consequently  assess  its 
internal  controls.  Maintenance  safety  audits  are  intended 
to  complement  other  safety-data  sources  such  as  ASRS 
and  M-ASAP  by  tapping  different  feedback  mechanisms 
and  by  identifying  hazards  before  they  become  events  or 
accidents. 

The  purpose  of  this  report  is  to  document  the  devel¬ 
opment  of  LOSA  in  flight  operations,  its  successes  and 
lessons  learned,  and  describe  the  extension  of  the  flight 
deck  version  of  LOSA  to  aviation  maintenance  and  ramp 
operations. 
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BACKGROUND 

LOSAdevelopmentwasinitiallystartedin  1991  at  the 
University  of  Texas  at  Austin  (UT-Austin)  with  funding 
from  the  FAA.  The  development  of  LOSA  stemmed  from 
a  request  by  Delta  Air  Lines  to  validate  the  operational  im¬ 
pact  of  its  three-day  Crew  Resource  Management  (CRM) 
training  course.  Analysts  soon  realized  that  existing  data 
collection  methods  did  not  assemble  adequate  informa¬ 
tion  regarding  flight  crew  adherence  to  standard  operating 
procedures  (SOPs)  and  environmental  influences  on  flight 
crew  performance.  To  explore  the  effectiveness  of  CRM 
training  transfer,  a  partnership  was  established  in  1994 
between  the  UT-Austin  Human  Factors  Research  Project 
and  Delta  Air  Lines.  The  goal  was  to  develop  a  line  audit 
methodology  utilizing  jump-seat  observations  on  regularly 
scheduled  flights  (i.e.,  LOSA).  In  its  early  form,  LOSA 
mostly  focused  on  CRM  performance  (Klinect,  Mur¬ 
ray,  Merritt,  &  Helmreich,  2003).  The  audits  provided 
actionable  data  about  strengths  and  weaknesses,  allowing 
prioritization  and  improvement  of  CRM  training.  They 
also  supported  the  validity  of  findings  from  the  CRM 
training  data.  Other  major  airlines  then  conducted  their 
own  CRM  audits  in  collaboration  with  UT-Austin. 

The  next  major  development  of  LOSA  evolved  from 
the  advancement  of  systems  thinking  and  human  error 
research  in  the  field  of  aviation  human  factors.  In  1 997,  the 
UT-Austin  team  collaborated  with  Continental  Airlines 
to  expand  the  method  to  focus  on  the  management  of 
common  threats  and  errors.  This  ultimately  evolved  into 
the  Threat  and  Error  Management  (TEM)  model  and 
the  creation  of  the  current  LOSA’s  underlying  theoreti¬ 
cal  framework.  Continental  Airlines  was  the  first  to  use 
a  TEM-based  LOSA  to  target  areas  for  improvement 
(e.g.,  pilots’  error  management  training).  In  1997  and 
1998,  the  UT-Austin  research  team  conducted  LOSAs 
at  three  airlines  (Klinect,  Wilhelm,  &  Helmreich,  1999). 
The  observers  documented  threats  (external  events  such 
as  adverse  weather  or  errors  originated  by  non-cockpit 
personnel),  recorded  flight  crew  errors,  and  rated  the 
crew  using  CRM  behavioral  markers  in  accordance  with 
TEM  performance.  Along  with  the  documented  threats 
and  errors,  observers  also  recorded  how  each  event  was 
managed  by  the  flight  crew.  Initial  data  showed  that  threats 
and  errors  are  common.  Their  types  and  occurrences 
varied  across  airlines.  Notably,  LOSA  data  illuminated 
the  behaviors  that  led  to  effective  and  ineffective  threat 
and  error  management.  The  nuances  included  in  this 
proactive  data  collection  strategy  populate  a  richer  and 
more  extensive  library  of  threats  and  errors  than  reactive 
accident/incident  reporting.  LOSA  examines  responses 
to  errors  that  have  not  yet  resulted  in  an  accident  or 
incident.  Capturing  effective  responses  allows  LOSA  data 


to  provide  insight  into  normal  flight  operations  and  aid 
training.  Follow-up  studies  showed  a  sizable  improve¬ 
ment  at  Continental  Airlines  in  safety  and  overall  crew 
performance  (Klinect,  et  ah,  2003). 

LOSA  data  collection  is  conducted  using  the  LOSA 
observation  form  under  strict  non-jeopardy  conditions, 
meaning  that  crews  are  not  at  risk  of  receiving  reprimands 
due  to  observed  actions.  Establishing  that  there  is  a 
non-punitive  policy  toward  errors  during  data  collection 
improves  the  validity  of  the  data  by  encouraging  those 
being  observed  to  carry  on  their  natural  work  behaviors. 

LOSA  was  first  operationally  deployed  as  an  Inter¬ 
national  Civil  Aviation  Organization  (ICAO) -endorsed 
safety  program  following  the  First  LOSA  Week,2  which 
was  hosted  by  Cathay  Pacific  Airways  in  March  200 1 . 
After  several  years  of  development  and  refinement, 
LOSA  has  evolved  into  a  strategy  to  provide  safety  data 
comprised  of  normal  operations  in  technical  and  human 
performance  areas.  The  LOSA  observations  provide  data 
to  develop  countermeasures  to  operational  threats  and 
errors  (ICAO,  2002).  It  has  since  been  used  and  validated 
by  many  international  airlines  and  is  now  recognized  as 
a  key  element  in  an  airline’s  Safety  Management  System 
(SMS).  It  also  provides  a  data-driven  mechanism  for 
measuring  change  (Veilette,  2008).  Based  on  the  success 
at  many  carriers  that  use  LOSA,  ICAO  made  LOSA  a 
central  focus  of  its  Flight  Safety  and  Human  Factors 
Program  and  endorsed  it  as  an  industry  best-practice  for 
normal  operations  monitoring.  The  FAA  also  approves 
LOSA  as  one  of  its  voluntary  safety  programs  (Merritt 
&  Klinect,  2006). 

UT-Austin  provided  “how-to”  guides  as  an  open 
source  through  numerous  conference  presentations  and 
papers  to  the  airline  industry  about  flight  deck  LOSA, 
as  well  as  details  about  why  and  how  to  set  up  a  LOSA. 
UT-Austin  helped  develop  two  primary  guidelines:  1) 
Line  Operations  Safety  Audit  (ICAO,  2002), 3  and  2) 
Advisory  Circular  120-90  Line  Operations  Safety  Audits 
(FAA).  Consequently, The  LOSA  Collaborative  (a private 
organization)  was  formed  in  the  interest  of  protecting  the 
collected  LOSA  data. 


2The  First  LOSA  Week,  a  pioneering  event  organized  by  the 
International  Civil  Aviation  Organization,  indicates  the  completion 
of  a  transformation  from  research  concepts  to  operational  tools. 
Since  then,  LOSA  evolved  and  extended  into  the  Normal  Operations 
Safety  Survey  (NOSS),  designed  for  air  traffic  control  operations, 
and  has  become  a  successful  and  acknowledged  contribution  to  the 
management  of  safety  (ICAO,  2008). 

3The  ICAO  LOSA  manual  is  outdated.  For  example,  the  hierarchical 
“sticks  and  boxes”  diagram  in  the  manual  implies  that  every  error 
has  a  threat,  which  was  found  to  not  be  true  through  actual  LOSA 
observations.  Most  errors  are  “spontaneous  errors”  without  any 
previous  threat  (].  Klinect,  personal  communication,  April  1,  2009). 
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LOSA  Operating  Characteristics 

The  ICAO  LOSA  (2002)  manual  described  10 
must-have  LOSA  operating  characteristics  (Table  1). 
These  characteristics  ensure  the  integrity  of  the  LOSA 
methodology  and  its  data.  (ICAO,  2002). 

Benefits  of  LOSA 

LOSA  does  not  rely  on  outcomes,  such  as  an  incident 
or  accident,  to  generate  data.  It  provides  a  unique  op¬ 
portunity  to  sample  all  activities  in  normal  operations, 
both  successful  and  unsuccessful,  by  noting  the  problems 
crews  encounter  and  how  they  manage  them  (FAA.2006). 

Proactive  approaches  are  aligned  with  the  principles 
of  risk  management  and  SMS.  Instead  of  focusing  on 
problems,  LOSA  offers  the  flexibility  of  observing  normal 
operations  (where  threats  will  always  exist)  and  target¬ 
ing  problematic  areas.  LOSA  is  a  project-based  process, 
which  includes  advance  planning,  observer  selection  and 
training,  data  collection,  analyses,  and  a  final  report. 
Repeating  a  LOSA  can  help  maintain  a  broad  focus  of 
an  earlier  LOSA  and  track  any  targeted  enhancements. 
ICAO  recommends  “to  sustain  safety  in  a  constantly 
changing  environment,  data  must  be  collected  and 
analyzed  on  a  routine  basis  to  identify  the  targets  for 
enhancement  and  then  a  formal  Safety  Change  Process 
(SCP)  to  bring  out  improvement”  (ICAO,  2002).  Prior 
to  programs  like  LOSA,  SCPs  were  based  on  findings 
from  incident/ accident  investigations,  experience,  and 
intuition.  Today,  SCPs  must  deal  with  the  precursors  of 
incidents/accidents  and  be  based  on  the  accumulated 
data  repository  and  interactive  detailed  analysis  methods 
generated  by  programs  like  LOSA. 

Flight  deck  LOSA  has  resulted  in  numerous  improve¬ 
ments  including  the  modification  of  dispatch  paperwork, 
reallocation  of  resources,  and  revision  of  procedures 
based  on  the  problems  uncovered  through  standardized 
observations  and  scientific  data  analyses.  LOSA  data  have 
greater  accuracy  than  anecdotal  observations  and  can  help 
answer  questions  about  problem  frequency,  surrounding 
conditions,  and  events  leading  up  to  an  issue.  The  data  also 
provide  better  detail  than  voluntary  reports  and  identify 
problematic  procedures  and  policies  by  highlighting  poor 
adherence  rates  (Vedette,  2008).  At  Continental  Airlines, 
an  airline-wide  LOSA  in  1 996  uncovered  that  pilots  were 
having  trouble  flying  uniform  approaches  to  company- 
defined  standards.  As  it  turned  out,  the  problem  was  not 
that  the  pilots  were  managing  approaches  ineffectively 
but  that  the  company’s  standards  were  ambiguous.  LOSA 
results  made  it  possible  to  convince  management  to 
modify  SOPs  for  approaches,  and  the  results,  verified 
through  a  LOSA  in  2000,  showed  a  59%  reduction  in 


nonconforming  approaches  (Croft,  2001).  In  addition, 
a  55%  decline  in  unstabilized  approaches  was  achieved 
by  the  company  as  a  result  of  training  developed  from 
the  LOSA  findings  (Tullo,  2002). 

Problems  With  LOSA  Implementation 

Lack  of  adherence  to  the  10  LOSA  characteristics, 
which  sometimes  occurs  in  internal  LOSA  programs, 
can  reduce  the  effectiveness  of  the  LOSA  audit  (ICAO, 
2002) .  First,  LOSA  results  are  not  always  shared  with  the 
pilots.  This  may  be  the  result  of  management  considering 
a  particular  LOSA  observation  result  “not  great  news”  and 
deciding  not  to  take  action.  A  potential  countermeasure 
to  this  problem  is  to  instill  the  concept  that  providing 
feedback  will  advance  future  LOSA  efforts  in  the  sense 
of  (a)  illustrating  that  pilots’  opinions  and  inputs  are 
taken  seriously  by  the  company  and  (b)  motivating 
observers  and  those  being  observed  to  participate  more 
collaboratively  in  future  studies.  Second,  airlines’  internal 
LOSA  programs  sometimes  specify  the  identities  of  the 
observers  and  those  being  observed.  This  is  a  problem 
because  potential  disclosure  of  identity  may  prevent 
observers  from  providing  honest  feedback.  Information 
such  as  name,  employee  identification  number,  flight 
number,  and  date  should  not  be  recorded  on  a  LOSA 
audit  form.  Departure/arrival  cities,  aircraft  type,  and 
pilot  role  are  the  only  demographic  information  that 
should  be  recorded.  Everything  possible  should  be  done  to 
encourage  anonymity.  Pilots’  trust  in  the  LOSA  program 
is  paramount,  and  any  violation  of  anonymity,  whether 
a  penalty  follows  or  not,  violates  that  trust.  Third,  some 
internal  LOSA  programs  logged  threats  and  errors  but  did 
not  describe  how  they  were  managed.  A  threat  or  error 
may  not  occur  frequently  but  may  still  be  poorly  man¬ 
aged  and  have  unwanted  outcomes.  This  highlights  the 
importance  of  LOSA  not  being  just  a  threats-and-errors 
counting  exercise;  the  management  of  these  threats  and 
errors  is  critical. 

McDonald  and  Fuller  (1994)  found  that  some 
organizations  focus  only  on  auditing  documentation, 
physical  resources,  and  infrastructure,  while  neglecting 
observations  of  operational  activities.  Audits  conducted 
by  external  agencies  and  internal  safety  departments  may 
prompt  altered  and  rehearsed  work  behaviors,  which 
potentially  lead  to  inaccurate  data.  LOSA  is  different 
but  complementary  to  other  proactive  safety  programs 
such  as  Flight  Operational  Quality  Assurance  (FOQA) 
and  ASAP  by  providing  a  “neutral,  third-party  perspec¬ 
tive”  (FAA,  2006).  Each  offers  unique  insight;  and 
used  together  they  can  aid  understanding  and  mitigate 
operational  risk. 
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Table  1.  LOSA  Characteristics  with  flight  deck  examples 


Characteristic 

Examples  in  Flight  Deck  LOSA 

1 .  Peer-to-peer  observations  during  normal 
operations 

•  Routine  flights  only  -  no  line  checks  or  training  flights 

•  No  debriefings  or  post-flight  interviews  asking  crews  to 
comment  on  their  errors  and/or  undesired  aircraft  states 

2.  Anonymous,  confidential,  and  non-punitive 
data  collection 

•  No  crew  names,  flight  numbers,  or  other  identifying 
information 

•  Observer  identity  kept  anonymous 

•  Data  used  for  safety  purposes  only,  not  disciplinary 
action 

3.  Voluntary  participation 

•  Flight  crews  have  the  right  to  decline  a  LOSA 
observation 

4.  Trusted  and  trained  observers 

•  Observer  selection  -  management/union  list  of 
candidates 

•  Diverse  observer  team  -  pilots,  check  airmen, 
instructors,  safety  experts,  members  of  human  factors 
groups,  external  observers 

•  Training  length  (5  days):  ground  school  (2),  test 
observations  (2),  &  recalibration  (1) 

•  Majority  should  be  regular  pilots  from  within  the  airline 

5.  Joint  management/union  sponsorship 

•  Steering  committee  -  flight  operations,  training,  safety, 
and  union 

•  Symbolized  with  a  signed  agreement  and  sent  to  all 
pilots 

6.  Systematic  observation  instrument  based 
on  TEM 

•  Safety-targeted  data  collection  form 

•  Observers  record  TEM  events  that  they  see  and/or  hear 
and  write  narratives  for  contextual  support 

7.  Secure  data  collection  repository 

•  Third  party  or  pilot  association  gate  keeper 

•  Pilots  must  believe  that  observations  will  not  be 
“misplaced”  or  improperly  disseminated 

8.  Data  verification  roundtables 

•  Three  to  five  representatives  from  various  parts  of  the 
airline  scan  the  raw  data  for  inaccuracies 

•  TEM  data  checked  for  coding  accuracy  and  consistency 
with  SOP 

•  On  completion,  data  analysis  begins 

9.  Data-derived  targets  for  enhancement 

•  Serve  as  benchmarks  for  organizational  change 

•  LOSA  adopts  a  “measure,  change,  measure  again” 
approach 

10.  Feedback  of  results  to  the  workforce 

•  LOSA  findings  and  information  on  how  airline 
management  intends  to  respond  to  the  findings  with 
organizational  change 
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Extension  to  Aviation  Maintenance  and  Ramp 
Operations 

There  remains  substantial  opportunity  for  safety  im¬ 
provement  on  the  ramp  and  in  the  hangar.  The  Flight 
Safety  Foundation  (Lacagnina,  2007)  estimated  that  the 
airline  industry  worldwide  was  losing  $5  billion  a  year 
in  direct  and  indirect  costs  associated  with  aircraft  dam¬ 
age  on  the  ramp.  It  was  further  estimated  that  243,000 
people  were  injured  on  the  ramp  every  year.  Thus,  we 
believe  that  additional  methods  of  reducing  damage  and 
injuries  are  imperative.  The  LOSA process  holds  promise 
as  a  means  of  reducing  the  incidents  and  accidents  in 
ramp  and  maintenance  operations  because  LOSA  enables 
ramp  and  maintenance  workers  to  identify  and  develop 
methods  to  address  threats  and  errors  before  they  lead 
to  an  incident  or  accident. 

Several  companies  have  instituted  LOSA  themed  pro¬ 
grams  aimed  at  reducing  maintenance  errors  and  ground 
operation  damage.  These  LOSA  programs  predate  the 
current  LOSA  effort  and  provide  many  valuable  lessons. 
The  development  of  numerous  subject  matter  experts 
(SMEs)  was  one  of  the  beneficial  outcomes  of  these  ef¬ 
forts:  they  provided  guidance  during  the  development 
of  the  current  LOSA  program. 

These  companies  have  also  experienced  marked  success 
as  a  result  of  their  efforts.  Continental  Airlines,  Delta 
Airlines,  and  Qantas  Airlines  reported  benefits  from  their 
LOSA  programs  and  are  listed  below. 

Continental  Airlines'* 

Ramp-LOSA  (R-LOSA).  In  2008,  among  447  prob¬ 
lems  identified  by  the  flight  operations  LOSA  at  COAir, 
147  (29%)  were  ground  safety  issues.  An  examination 
of  flight  operations  LOSA  archival  data  revealed  that 
the  industry  average  is  only  16%  for  flight  operations 
ground  safety  issues. 

To  improve  ground  safety  performance,  COAir  es¬ 
tablished  several  safety  programs  under  the  umbrella  of 
its  SMS — for  example,  the  Safety  Recognition  Program 
and  R-LOSA.  Station  #1  had  the  same  ground  safety 
programs  as  Station  #2  but,  Station  #1  added  the  R- 
LOSA  program  in  2007.  Both  stations  improved  their 
group  safety  performance  dramatically  over  a  three-year 
span  (2006-2009) .  Data  for  2009  are  only  available  from 


4The  information  in  this  section  is  based  on  a  site  visit  to  George 
Bush  Intercontinental  Airport  on  March  18,  2008  and  personal 
communications  with  Doc  Garrett  (Senior  Manager,  Maintenance 
Human  Factors,  Logistics  &  GSE  Systems,  Tech  Ops) ;  Rodney  Luetzen 
(Managing  Director,  Reliability);  Gerry  McGill  (Regional  Manager, 
Safety  &  Regulatory  Compliance,  Flight  Ops);  and  Guy  Schroeder 
(Director  Ground  Safety,  Safety  &  Regulatory  Compliance)  between 
March  2008  and  April  2009. 


January  through  October.  Monthly  averages  for  the  first 
10  months  were  used  to  estimate  November  and  Decem¬ 
ber  2009  ground  damage  mishaps,  and  consequently,  the 
averages  for  the  entire  year.  However,  the  improvement 
observed  by  Station  #  1  is  more  than  Station  #2,  which  can 
potentially  be  attributed  to  the  effectiveness  of  R-LOSA 
program  (Note  that  Station  #l’s  initial  safety  performance 
was  better  than  Station  #2).  Ground  safety  performance 
was  assessed  using  three  measures:  (1)  ground  damage 
mishaps  (total  number  of  occurrences),  (2)  ground  dam¬ 
age  mishaps  (mishap  rate  per  10,000  departures),  and  (3) 
cost  of  ground  damage  mishaps. 

Ground  operation  mishaps  can  further  be  categorized 
as  attributable  mishaps  and  non-attributable  mishaps. 
Attributable  mishaps  are  a  result  of  human  error  and 
are  charged  back  to  the  responsible  department  or  ven¬ 
dor.  Non-Attributable  Mishaps  include  Foreign  Object 
Damage  (FOD).  The  costs  are  not  recovered  for  these 
mishaps.  Both  stations  showed  a  dramatic  decrease  in  the 
total  number  of  ground  damage  mishaps  from  2006  to 
2009.  The  number  of  attributable  and  non-attributable 
mishaps  for  Station  #1  dropped  73%  and  85%,  re¬ 
spectively,  whereas  the  drops  for  Station  #2  were  58% 
and  67%,  respectively.  The  cost  of  ground  damage  also 
decreased  overall  between  the  years  of  2006  and  2009  in 
both  stations.  However,  the  cost  of  attributable  mishaps 
for  Station  #1  increased  very  slightly  in  2008. 

For  Station  #1,  the  ground  damage  mishap  rate  also 
decreased  significantly  from  2006  to  2009:  per  10,000 
departures,  attributable  mishaps  dropped  61%  and  non- 
attributable  mishaps  dropped  to  zero.  For  Station  #2,  both 
attributable  and  non-attributable  mishap  rates  decreased 
from  2006  to  2009  (43%  and  45%,  respectively).  The 
cost  of  ground  damage  also  decreased  overall  between 
the  years  of  2006  and  2009  in  both  stations.  The  most 
significant  improvements  were  observed  in  the  following 
four  areas:  ground  handling  operations,  struck  by  vehicle 
in  motion,  taxi-tow-push,  and  maintenance  operations. 

Maintenance  LOSA  (M-LOSA).  M-LOSA  findings 
help  make  deactivation  procedures  more  workable, 
efficient,  and  safer.  As  an  example,  B767  leading  edge 
device  deactivation  and  reactivation  procedures  used 
to  take  three  hours  to  properly  lock  out  and  tag  out5 
without  individual  sign-offs.  An  M-LOSA  auditor  identi¬ 
fied  this  inefficiency,  which  was  then  addressed  by  Tech 
Publications  by  rewriting  their  deactivation/reactivation 
procedures.  Previously,  the  lockout  and  tag  out  process 


5“Lock  out  and  tag  out”  refers  to  specific  practices  and  procedures  to 
safeguard  employees  from  the  unexpected  energization  or  startup  of 
machinery  and  equipment,  or  the  release  of  hazardous  energy  during 
service  or  maintenance  activities. 
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involved  unnecessary  deacdvation  of  some  systems  follow¬ 
ing  a  37-page  procedure.  Some  steps  required  personnel 
to  repeatedly  reference  different  sections  of  the  manual 
and  there  were  no  individual  sign-offs  when  following 
the  manual  (e.g.,  deactivate  the  slats  per  AMM  27).  The 
new  workcard  is  2-pages  long  with  clearly  defined  steps. 
Now,  with  individual  sign-offs,  this  modified  process 
takes  between  30  and  45  minutes  to  complete.  The  new 
standardized  procedures  also  help  to  avoid  problems 
caused  by  shift  changes  (deactivation  and  reactivation 
are  often  carried  out  on  different  shifts)  and  interrup¬ 
tions.  This  deactivation/reactivation  procedure  has  been 
implemented  in  the  entire  Continental  Airlines  fleet. 
Because  of  the  changes  implemented  by  M-LOSA,  the 
threats  have  been  reduced  tremendously,  and  no  damage 
to  the  aircraft  has  occurred  at  the  time  of  this  publica¬ 
tion. 

Delta  Air  Lines6 

Due  to  ground  operations  safety  concerns,  the  Delta 
leadership  team  made  several  requests  to  the  Atlanta 
Airport  Authority  asking  that  they  repaint  the  clearance 
lines  in  the  international  concourse.  However,  the  requests 
were  ignored  until  Delta  presented  the  results  of  a  Ramp 
Operations  Safety  Audit  (ROSA)  at  an  airport  operator 
meeting.  ROSA  is  considered  an  effective  communication 
tool  and  a  critical  component  of  Deltas  SMS.  ROSA  data 
are  reactive  in  addressing  existing  problems  and  proactive 
in  helping  the  leadership  team  form  goals  and  objectives 
with  a  reasonable  timeline. 

The  ROSA  data  illustrated  serious  problems  caused  by 
the  missing  clearance  lines.  The  Atlanta  Airport  Authority 
was  convinced  of  the  urgency  in  repainting  the  clearance 
lines  by  the  ROSA  data.  Following  repainting,  ground 
equipment  operators  have  consistently  obeyed  the  rule 
of  parking  outside  the  clearance  lines  when  airplanes  are 
not  at  the  gate.  This  practice  has  been  consistently  imple¬ 
mented  across  six  different  concourses.  Consequently, 
parking  violation-induced  ground  equipment  damage 
and  occurrence  of  FOD  on  the  ramp  have  decreased.  The 
ready  availability  of  the  equipment  has  also  improved 
significantly. 


Qantas  Airways7 

In  January  2008,  Qantas  Airways  successfully 
conducted  its  first  Ground  Operational  Safety  Audit 
(GOSA),  an  adaptation  of  the  airlines’  long-established 
LOSA  methodology  to  the  ramp  environment.  GOSA 
was  used  to  observe  the  behavior  of  ramp  teams  during 
aircraft  turnarounds  and  provided  quantitative  data  on 
the  threats,  errors,  and  undesirable  operational  states  that 
threatened  the  operational  safety  of  ground  operations. 
GOSA  provided  Qantas  ramp  management  a  means 
of  gathering  data  on  strengths  and  weaknesses  of  the 
operation,  interface  problems,  effectiveness  of  training, 
quality  and  usability  of  procedures,  and  a  rationale  for 
resource  allocation.  It  has  also  provided  quantitative  and 
qualitative  data  on  the  processes  undertaken  by  staff  that 
result  in  work  shortcuts,  injury,  or  risk  to  other  staff. 
The  implementation  of  GOSA  has  resulted  in  positive 
tangible  outcomes  for  Qantas.  Many  simple  day-to-day 
procedures  have  been  adapted  to  reflect  the  results  of  the 
audits.  This  has  had  the  beneficial  effects  of  streamlining 
ramp  practices  and  contributing  to  staff  engagement. 

GOSA  has  allowed  Qantas  to  gather  data  on  the 
work  practices  of  external  ramp  service  providers,  and 
subsequently  work  with  those  providers  to  eradicate 
ineffective  procedures.  Qantas  was  then  able  to  further 
satisfy  its  customers  by  ensuring  compliant,  efficient,  and 
cohesive  ramp  service. 

Air  Transport  Association  (ATA)  Human  Factors 
Task  Force 

In  December  2008,  a  group  of  FAA  and  Saint  Louis 
University  researchers  began  collaborating  with  the  Air 
Transport  Association  (ATA)  Maintenance  and  Ramp 
Human  Factors  Task  Force  (ATA  HF  Task  Force).  The 
Task  Force  developed  M-LOSA  and  R-LOSA  forms, 
training  documentation,  and  the  base  structure  for  data 
warehousing  and  reporting.  Development  progressed 
through  numerous  consultations,  combined  with  iterative 
development,  testing,  and  refinement.  The  development 
of  the  maintenance  and  ramp  LOSA  forms,  procedures, 
and  software  involved  a  core  team  of  approximately  30 
experts  from  the  maintenance,  ramp,  and  human  factors 
communities.  The  team  produced  more  than  20-line, 
base  maintenance,  and  ramp  operations  forms  designed 
for  use  in  various  LOSA  audits  (see  sample,  Appendix  A). 


6The  information  in  this  section  is  based  on  personal  communication  7The  information  in  this  section  is  based  on  personal  communication 

with  Mr.  Alex  Vargas,  Manager  of  Aviation  Safety,  Delta  Airlines  with  ShaunTrimby,  Coordinator  Human  Factors  and  Safety  Programs, 

ROSA  (A.  Vargas,  personal  communication,  February  19,  2009).  Qantas  Airways  (S.Trimby,  personal  communication,  March  5, 2009). 
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Form  Development 

The  ATA  HF  Task  Force  found  that  forms  comprising 
a  combination  of  checklists  and  comments  would  be  more 
effective  for  this  domain,  rather  than  the  narrative  method 
used  with  the  flight  deck  LOSA.  The  forms  and  checklists 
were  constructed  to  reflect  the  procedures  followed  by 
maintenance  and  ramp  workers.  Maintenance  forms  are 
based  on  procedures  such  as  Troubleshooting  and  Prepare 
to  Install  with  ramp  forms  based  on  procedures  such  as 
Downloading&nd  Uploading.  The  line  items  on  the  forms 
follow  the  general  flow  of  activities  found  during  each 
procedure.  This  makes  it  easier  for  a  LOSA  observer  to 
locate  where  a  particular  item  should  be  recorded.  If  the 
observer  encounters  activities,  threats,  and  errors  that  are 
not  encompassed  by  the  forms,  he  or  she  is  instructed 
to  address  these  items  in  detail  in  the  general  comments 
for  that  form. 

The  most  common  categories  of  threats  and  errors 
(e.g.,  fatigue,  incorrect  maintenance  manual)  are  assigned 
codes.  The  codes  provide  the  ability  to  query  and  analyze 
the  data  more  quickly  and  in  ways  not  possible  (e.g.,  in¬ 
ferential  statistics)  with  a  largely  narrative  approach.  The 
comments  provide  additional  detail  that  can  be  accessed 
as  needed  but  come  with  the  drawback  of  requiring  more 
time  to  read,  comprehend,  and  interpret  the  information. 

Beta  Testing 

After  initial  development  of  the  forms,  beta  testing  al¬ 
lowed  input  from  more  than  100  maintenance  technicians 
and  ramp  personnel.  We  conducted  beta  tests  for  ramp, 
line  maintenance,  and  base  maintenance  at  numerous 
locations  across  the  United  States.  The  task  force  selected 
Part  135  and  Part  121  carriers  representing  both  passenger 
and  cargo  operations  for  beta  testing.  For  each  beta  test, 
a  team  of  10-15  experts  were  deployed.  LOSA  trainers 
preceded  the  team  to  prepare  the  maintenance  and  ramp 
workers  for  being  observed  and  to  train  a  carefully  chosen 
team  to  conduct  the  initial  observations.  The  instructors 
provided  information  on  the  basics  of  LOSA,  including 
the  confidential,  non-jeopardy  characteristics  of  the  ob¬ 
servations,  as  well  as  detailed  training  on  how  to  conduct 
a  LOSA  including  the  recording  and  coding  of  data. 

The  task  force  beta  testing  team  remained  on  site  and 
answered  questions  and  recorded  feedback  throughout  the 
initial  weeks  of  testing.  Following  each  shift,  we  debriefed 
and  addressed  questions  and  captured  lessons  learned  on 
LOSA  procedures  and  checklist  content.  The  task  force 
discussed  the  lessons  learned  at  ensuing  meetings  and 
made  changes  as  necessary. 


Database  Development 

Databases  were  created  to  streamline  audit  information. 
Currently,  users  enter  data  collected  from  audits  into  an 
Access®  database  for  future  analyses  and  reporting.  Efforts 
are  underway  to  develop  a  more  robust  and  powerful 
software  tool  based  on  Structured  Query  Language.  The 
tool  under  development  will  allow  LOSA  teams  to  enter 
the  data  from  a  virtually  unlimited  number  of  LOSA 
observations.  The  new  system  will  also  allow  connec¬ 
tions  from  a  greater  number  of  observers,  stations,  and 
organizations.  These  factors  are  critical  as  the  numbers 
of  observations  are  anticipated  to  rapidly  extend  into 
the  thousands. 

Training 

Training  was  developed  to  ensure  effective  implemen¬ 
tation  of  the  LOSA  program  that  described  the  purpose 
of  LOSA,  theoretical  foundation  (TEM  model),  how  to 
conduct  a  LOSA  via  the  checklist  forms,  and  data  man¬ 
agement.  Computer-based  training  via  scenario-based, 
guided  presentation  allows  companies  the  flexibility  to 
introduce  the  basics  of  LOSA  while  considering  practi¬ 
cal  examples.  The  training  materials  provide  the  neces¬ 
sary  background  to  prepare  LOSA  observers  who  have 
no  background  in  TEM  or  LOSA.  The  initial  training 
module  provides  an  introduction  to  threat  and  error 
management,  how  it  relates  to  the  maintenance  or  ramp 
environment,  and  the  initial  foundation  for  LOSA.  The 
second  module  in  the  training  provides  detailed  informa¬ 
tion  on  LOSA,  how  and  why  it  was  developed,  previous 
successes,  and  what  it  means  for  ramp  and  maintenance 
workers.  The  third  and  final  module  provides  the  ob¬ 
servers  with  scenario-based  practice.  The  scenarios  allow 
the  observer  to  experience  a  distilled  version  of  several 
real-world  observations,  practice  recording  the  data,  and 
the  chance  to  review  what  LOSA  experts  have  recorded 
for  each  scenario. 

The  task  force’s  goal  was  to  develop  a  practical,  cus¬ 
tomizable,  and  scalable  methodology  and  deliver  it  to 
the  industry  as  a  part  of  a  freely  available  toolset.  The 
culmination  of  that  goal  was  realized  when  the  ATA  L1F 
Task  Force  released  the  M-LOSA  and  R-LOSA  forms, 
procedures,  software,  and  training  materials  for  the  public 
on  the  Internet  (https://hfskyway.faa.gov/LlFSkyway/ 
L  O  S  A  H  o  m  e .  as  p  x ) . 
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CONCLUSIONS 

This  report  provides  a  review  of  rhe  development  and 
implementation  of  flight  deck  LOSA,  as  well  as  description 
of  attempts  to  transit  LOSA  to  aviation  maintenance  and 
ramp  operations.  The  R-LOSA  and  M-LOSA  methodolo¬ 
gies  aim  to  use  pre-identified  visible  precursors  to  ramp  or 
maintenance  events,  thus  ensuring  an  efficient,  reliable, 
and  valid  audit  of  normal  activity. 

Precursors  may  lay  hidden  for  years  waiting  for  the 
chance  to  team  up  with  other  factors  to  cause  an  inci¬ 
dent.  The  R-LOSA  and  M-LOSA  audits  are  expected  to 
encourage  behavior  change  in  ramp  and  maintenance 
operations  and  allow  sub-units  of  an  organization  to 
build  in  some  flexibility  to  address  their  key  problems 
and  conquer  them  one  at  a  time.  The  periodic  audits 
can  help  ensure  that  specific  problems  identified  have 
been  resolved,  as  well  as  assess  the  effectiveness  of  safety 
recommendations. 

The  development  of  R-LOSA  and  M-LOSA  will 
build  upon  existing  knowledge  regarding  safety  across 
high-consequence  industries.  In  particular,  the  impact  of 
observation  of  normal  behaviors  in  the  aircraft  mainte¬ 
nance  and  ramp  operations  will  help  qualify  and  quantify 
the  efforts  made  by  aircraft  mechanics  and  ramp  agents 
to  prevent  or  reduce  incidents  and  accidents. 
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Appendix  A 

Sample  LOSA  Observation  Form 

Install 


Threat  Codes  Legend 

T/A.  Information 

T/G.  Environment  /  Facilities 

T/B.  Equipment  /  Tools  /  Safety  Equipment 

T/H.  Organizational  Factors 

T/C.  Aircraft  Design  /  Configuration  /  Parts 

T/I.  Leadership  /  Supervision 

T/D.  Job  /  Task 

T/J.  Communication 

T/E.  Knowledge  /  Skills 

T/K.  Quality  Control 

T/F.  Individual  Factors 

T/L.  Other  Contributing  Factors 

Observation  Number: _  DDid  not  observe  this  section 


Safety  Risk 

N/A, 

Safe  (S), 

At  Risk  (AR), 
Didn't 
Observe 
(DNO) 

Threat 

Code 

(See 

Threat 

Codes 

List) 

Threat 

Effectively 

Managed 

Y/N 

Error  Outcome 

1  .Inconsequential 

2.  Undesired  state 

3. Additional  error 

& 

Remarks 

Safety 

1 

Notes,  cautions,  and  warnings 
reviewed 

2 

Notes,  cautions,  and  warnings 
followed 

Personnel 

3 

Required  personnel  available 

Procedures 

4 

Effectivity/configuration 

verified 

5 

Materials  utilized 

6 

Servicing  procedures  followed 

7 

Installation  procedures 
followed 

Describe  the  threat(s).  How  did  the  technician(s)  manage  or  mismanage  the  threat(s)? 

Describe  the  technician  error(s)  and  associated  undesired  states 

Comments  -  Good  or  bad  (Please  provide  examples) 

Al 


